Biological information detection device equipped with light source projecting at least one dot onto living body

ABSTRACT

A biological information detection device includes first and second light sources, an image capturing system, and an arithmetic circuit. The first and second light sources project, onto a living body, at least one first dot formed by first light and at least one second dot formed by second light, respectively. The image capturing system includes first photodetector cells and second photodetector cells. The image capturing system generates and outputs a first image signal and a second image signal. The arithmetic circuit generates information concerning the living body, by using the first and second image signals. The first light includes fifth light. The second light includes sixth light. Each of the fifth light and the sixth light has a wavelength in a range from 650 nm to 950 nm. The wavelength of the sixth light is longer than that of the fifth light by 50 nm or more.

BACKGROUND

1. Technical Field

The present disclosure relates to biological information detectiondevices and, more particularly, to a biological information detectiondevice, a biological information detection method, and a psychologicalchange detection method for detecting biological information, such asheartbeat, in a non-contact manner.

2. Description of the Related Art

Heartbeat, blood flow, blood pressure, and blood oxygen saturation arewidely used as basic parameters for determining the health condition ofa person. These pieces of biological information relating to blood aretypically measured by contact-type measuring instruments. Sincecontact-type measuring instruments are attached to the body of asubject, measurement, especially, long continuous measurement, sometimesincurs the subject's discomfort.

Various attempts have been made to easily measure basic biologicalinformation for determining the health condition of a person. Forexample, Japanese Unexamined Patent Application Publication No.2005-218507 discloses a method for detecting heart rate in a non-contactmanner on the basis of image information of a face or the like obtainedwith a camera. Japanese Unexamined Patent Application Publication(Translation of PCT Application) No. 2003-517342 discloses a method formeasuring, using a white light source and a laser light source, bloodoxygen saturation on the basis of a laser Doppler effect of laser lightscattered behind the surface of a living body. Japanese UnexaminedPatent Application Publication (Translation of PCT Application) No.2014-527863 discloses a method for measuring, using an ordinary colorcamera, blood oxygen saturation while removing the influence of ambientlight.

In addition, many methods for estimating a psychological change of aperson have been proposed. For example, Japanese Unexamined PatentApplication Publication Nos. 6-54836 and 2008-237244 disclose methodsfor detecting, with thermography, a decrease in temperature at a noseportion that occurs when a person feels stress (nervous) orconcentrates.

SUMMARY

In one general aspect, the techniques disclosed here feature abiological information detection device including: a first light sourcethat, in operation, projects, onto a living body, at least one first dotformed by first light; a second light source that, in operation,projects, onto the living body, at least one second dot formed by secondlight; an image capturing system including first photodetector cellsthat, in operation, detect third light from the living body on which theat least one first dot is projected, and second photodetector cellsthat, in operation, detect fourth light from the living body on whichthe at least one second dot is projected, the image capturing system, inoperation, generating and outputting a first image signal and a secondimage signal, the first image signal denoting a first image of theliving body on which the at least one first dot is projected, the secondimage signal denoting a second image of the living body on which the atleast one second dot is projected; and an arithmetic circuit that, inoperation, generates information concerning the living body by using thefirst image signal and the second image signal output from the imagecapturing system and outputs the generated information, wherein: thefirst light includes fifth light having a wavelength longer than orequal to 650 nm and shorter than or equal to 950 nm; the second lightincludes sixth light having a wavelength longer than or equal to 650 nmand shorter than or equal to 950 nm; and the wavelength of the sixthlight is longer than the wavelength of the fifth light by 50 nm or more.

It should be noted that general or specific embodiments may beimplemented as an element, a device, a system, a method, or anyselective combination thereof.

Additional benefits and advantages of the disclosed embodiments willbecome apparent from the specification and drawings. The benefits and/oradvantages may be individually obtained by the various embodiments andfeatures of the specification and drawings, which need not all beprovided in order to obtain one or more of such benefits and/oradvantages.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a diagram for explaining a basic concept of how biologicalinformation is obtained in an embodiment of the present disclosure;

FIG. 1B is a diagram for explaining characteristics of an image of thesurface of a living body, obtained with a camera;

FIG. 2 is a diagram illustrating an absorption coefficient and ascattering coefficient of hemoglobin, melanin, and water, which are maincomponents of a living body, in a wavelength range from visible light tonear-infrared light;

FIG. 3A is a diagram illustrating a configuration of a biologicalinformation detection device according to a first embodiment;

FIG. 3B is a diagram illustrating an example of a configuration of acamera and an example of an output image in accordance with the firstembodiment;

FIG. 3C is a block diagram illustrating a configuration of a computer inaccordance with the first embodiment;

FIG. 4A is a diagram illustrating a first example to which a humandetection method according to the first embodiment is applied;

FIG. 4B is a diagram illustrating a second example to which the humandetection method according to the first embodiment is applied;

FIG. 5 is a diagram illustrating a contrast calculation method inaccordance with the first embodiment;

FIG. 6A is a diagram illustrating an example of a configuration and anexample of biological information (heart rate) generated in accordancewith a second embodiment;

FIG. 6B is a flowchart illustrating a flow of an image processingprocess in accordance with the second embodiment;

FIG. 7 is a diagram illustrating a configuration of a biologicalinformation detection device according to a third embodiment;

FIG. 8 is a diagram illustrating an overview of biological informationsensing using two cameras in accordance with the third embodiment;

FIG. 9 is a diagram illustrating transmittance characteristics of twobandpass filters in accordance with the third embodiment;

FIG. 10 is a diagram illustrating an example of pulse waves measuredwith a method according to the third embodiment;

FIG. 11 is a diagram illustrating results obtained by measuring bloodoxygen saturation with the method according to the third embodiment anda method of the related art;

FIG. 12 is a diagram illustrating a configuration of astereo-camera-type biological information detection device according toa fourth embodiment;

FIG. 13 is a diagram illustrating a configuration of a stereo-lens-typebiological information detection device according to a fifth embodiment;

FIG. 14A is a diagram illustrating a result obtained by performingstress sensing using the biological information detection deviceaccording to the fifth embodiment;

FIG. 14B is a diagram illustrating a nose portion and a cheek portion inan image in accordance with the fifth embodiment;

FIG. 14C is a diagram illustrating a change in blood flow and a changein blood oxygen saturation that are obtained using the biologicalinformation detection device according to the fifth embodiment;

FIG. 15 is a cross-sectional view schematically illustrating aconfiguration of a biological information detection device according toa sixth embodiment;

FIG. 16A is a diagram illustrating a nose portion and a forehead portionin an image in accordance with the sixth embodiment;

FIG. 16B is a diagram illustrating a change in total blood flow(oxyhemoglobin and deoxyhemoglobin) in time and a change inoxyhemoglobin blood flow (oxygen saturation) in time when an emotioncausing laughter is induced in accordance with the sixth embodiment;

FIG. 17A is a diagram schematically illustrating a configuration of abiological information detection device according to a seventhembodiment;

FIG. 17B is a diagram illustrating a plurality of color filters inaccordance with the seventh embodiment;

FIG. 17C is a diagram illustrating an example of images generated inaccordance with the seventh embodiment;

FIG. 18A is a diagram illustrating a configuration of a biologicalinformation detection device according to an eighth embodiment;

FIG. 18B is a diagram illustrating a plurality of color filters inaccordance with the eighth embodiment;

FIG. 18C is a diagram illustrating an example of images generated inaccordance with the eighth embodiment; and

FIG. 18D is a diagram illustrating an example of a configuration of amulti-spectral sensor including four types of color filters of R, G, B,and IR.

DETAILED DESCRIPTION

Underlying Knowledge Forming Basis of the Present Disclosure

Prior to description of embodiments of the present disclosure,underlying knowledge forming the basis of the present disclosure will bedescribed.

As described above, various attempts have been made to measure basicbiological information for determining the health condition of a person.For example, a method for detecting heart rate in a non-contact manneron the basis of image information of a face or the like obtained with acamera has been proposed in Japanese Unexamined Patent ApplicationPublication No. 2005-218507. In the method according to JapaneseUnexamined Patent Application Publication No. 2005-218507, heart rate isdetermined by analyzing a spatial frequency component of an obtainedcolor image. However, since the accuracy achieved by this methoddecreases due to the influence of disturbance light, such as lightilluminating a room, stable detection is difficult.

Pulse oximeters are commonly used to measure blood oxygen saturation.Pulse oximeters radiate two wavelengths of light in a red tonear-infrared wavelength range onto a finger inserted therein andmeasure transmittance of the light. In this way, pulse oximeters areable determine a ratio between an oxyhemoglobin concentration and adeoxyhemoglobin concentration in blood. Pulse oximeters are capable ofmeasuring blood oxygen saturation with a simple configuration. However,since pulse oximeters are contact-type devices, they may make peoplefeel restrained.

Japanese Unexamined Patent Application Publication (Translation of PCTApplication) No. 2003-517342 discloses an example of a non-contact-typeblood oxygen saturation measuring device. This device measures, using awhite light source and a laser light source, blood oxygen saturation onthe basis of a laser Doppler effect of laser light scattered behind thesurface of a living body. This method, however, makes a configuration ofthe device complicated, and the resulting signal is weak.

In Japanese Unexamined Patent Application Publication (Translation ofPCT Application) No. 2014-527863, a method for measuring, using anordinary color camera, blood oxygen saturation while removing theinfluence of ambient light has been proposed. Since this method isgreatly influenced by reflected light reflected by the surface of skin,it is difficult to measure blood oxygen saturation stably at a highaccuracy.

As described above, non-contact-type blood oxygen saturation measuringmethods of the related art have issues related to the accuracy and thestability. There is substantially no non-contact-type blood oxygensaturation measuring device that is put into practical use at present.

On the other hand, many methods for estimating, using thermography, apsychological change of a person have been proposed (for example,Japanese Unexamined Patent Application Publication Nos. 6-54836 and2008-237244). In these methods, a decrease in temperature at a noseportion is detected with thermography. Since lots of arteriovenousanastomosis is located at the nose portion of a person, bloodcirculation is easily disturbed by the influence of the autonomicnervous system. A psychological change, such as stress or nervousness,influences the autonomic nervous system. The influenced autonomicnervous system causes a decrease in blood flow at the nose portion,which then causes a decrease in temperature at the nose portion. Thedevices disclosed in Japanese Unexamined Patent Application PublicationNos. 6-54836 and 2008-237244 detect such a change in temperature withthermography, thereby estimating a psychological change of the subject.

Methods using thermography have a low responsivity because it takes timefor temperature to decrease and are influenced by environmentaltemperature. It is considered that a psychological change estimatingmethod that has a high responsivity and is more resistant to theinfluence of environmental temperature can be established if blood flowat the surface of the face can be accurately measured. However, noreliable method for measuring blood flow under skin at a high accuracyin a non-contact manner is established at present. Thus, methods havingthe above-described issues and using an expensive device such asthermography are predominant.

The inventor has focused on the above-described issues and has studied aconfiguration for addressing the issues. The inventor consequently hasfound out that the issues can be addressed by obtaining an image using alight source that projects at least one dot onto the surface of a livingbody and by separating, through signal processing, a component relatingto directly reflected light in the image from a component relating todiffused light caused in the living body. A biological informationdetection device according to an aspect of the present disclosureincludes at least one light source that, in operation, projects at leastone dot formed by light onto a target including a living body, an imagecapturing system that includes photodetector cells and that, inoperation, generates and outputs an image signal denoting an image ofthe target on which the at least one dot is projected, and an arithmeticcircuit that is connected to the image capturing system and, inoperation, processes the image signal output from the image capturingsystem. The arithmetic circuit may, in operation, generate and outputinformation concerning the living body by using the image signal. Such aconfiguration allows biological information to be obtained at a highaccuracy.

Principle

A principle allowing a biological information detection device to obtainbiological information at a high accuracy will be described below.

FIG. 1A is a diagram illustrating a schematic configuration of abiological information detection device according to an illustrativeembodiment of the present disclosure. The biological informationdetection device includes a light source 1 and a camera 2, which is animage capturing system. The light source 1 is an array point lightsource that projects a plurality of discretely arranged points (alsoreferred to as “arrayed points” or “dot pattern” herein) onto a targetincluding a living body 3. The light source 1 is arranged such that aplurality of points are projected onto the living body 3. The camera 2includes an image sensor, captures an image of a living-body surface 4,and generates and outputs an image signal.

FIG. 1B is a diagram for explaining characteristics of the image of theliving-body surface 4, obtained by the camera 2. Outgoing light LO fromthe light source 1 is reflected by the living-body surface 4. Surfacereflected light L1 reflected by the living-body surface 4 maintains animage of the arrayed points formed by the light source 1. In contrast,inside-body scattered light L2 that exits from the living-body surface 4after entering the living body 3 and being scattered inside the livingbody 3 no longer maintains the image of the arrayed points formed by thelight source 1 because of strong scattering caused inside the livingbody 3. The use of the light source 1 allows the surface reflected lightL1 and the inside-body scattered light L2 to be spatially separated fromeach other easily.

The living body 3 illustrated in FIG. 1A represents human skin andincludes epidermis 33, dermis 34, and a subcutaneous tissue 35. No bloodvessel is located at the epidermis 33, whereas a capillary 31 and anarteriole/venule 32 are located at the dermis 34. Since there is noblood vessel at the epidermis 33, the surface reflected light L1 doesnot contain information relating to blood (hereinafter, referred to asblood-related information). Since the epidermis 33 includes melanin thatstrongly absorbs light, the surface reflected light L1 reflected by theepidermis 33 becomes noise when blood-related information is obtained.Thus, the surface reflected light L1 is not only useless to obtainblood-related information but also disturbs acquisition of accurateblood-related information. Biological information can be detected at ahigh accuracy by suppressing the influence of the surface reflectedlight and by efficiently obtaining information of the inside-bodyscattered light.

To address the issues described above, embodiments of the presentdisclosure has a novel configuration with which directly reflected lightand inside-body scattered light are spatially separated using an arraypoint light source and an image capturing system. With this novelconfiguration, information concerning the living body can be measured ata high accuracy in a non-contact manner.

In the related art, methods using polarizing illumination such as theone disclosed in Japanese Unexamined Patent Application Publication No.2002-200050 have been used to separate directly reflected lightreflected by the living-body surface. In such methods using polarizingillumination, a polarizer having a polarized light transmission axisperpendicular to a polarization direction of illumination lightreflected by an image-capturing target is used. The influence of surfacereflected light can be suppressed by capturing an image with a camerathrough such a polarizer. However, since the degree of polarization ofsurface reflected light reflected by an uneven surface such as skinchanges depending on the position, separation of such directly reflectedlight is not sufficient. With a method according to an aspect of thepresent disclosure, the influence of surface reflected light can beeffectively suppressed since directly reflected light and scatteredlight is successfully spatially separated.

In the biological information detection device according to anembodiment of the present disclosure, the wavelength of light emitted bythe light source can be set to be, for example, longer than or equal toapproximately 650 nm and shorter than or equal to approximately 950 nm.This wavelength range is within a wavelength range of red tonear-infrared light. Herein, the term “light” is used not only forvisible light but also for infrared. The above wavelength range iscalled “optical tissue window” and is known as a range in whichabsorbance in the body is low.

FIG. 2 is a diagram illustrating wavelength dependency of a lightabsorption coefficient and an inside-body light scattering coefficientfor oxyhemoglobin, deoxyhemoglobin, melanin, and water. Light isabsorbed mainly by blood (i.e. hemoglobin) in a visible light range of650 nm or shorter, whereas light is absorbed mainly by water in awavelength range longer than 950 nm. Therefore, light in thesewavelength ranges is not suitable for obtaining biological information.In contrast, in a wavelength range from approximately 650 nm toapproximately 950 nm, the absorption coefficients for hemoglobin andwater are relatively low, and the scattering coefficient is relativelyhigh. Therefore, light of this wavelength range returns to the bodysurface after entering the body and being strongly scattered.Accordingly, with light of this wavelength range, biological informationcan be efficiently obtained.

A biological information detection device according to an embodiment ofthe present disclosure mainly utilizes light of this wavelength rangecorresponding to the “optical tissue window”. With this configuration,since the biological information detection device is able to separateand detect light directly reflected by the living-body surface andreturning light that has been scattered inside the living body, it canefficiently obtain biological information.

The present disclosure includes, for example, aspects recited in thefollowing items.

[Item 1] A biological information detection device includes:

a first light source that, in operation, projects, onto a living body,at least one first dot formed by first light;

a second light source that, in operation, projects, onto the livingbody, at least one second dot formed by second light;

an image capturing system including

-   -   first photodetector cells that, in operation, detect third light        from the living body on which the at least one first dot is        projected, and    -   second photodetector cells that, in operation, detect fourth        light from the living body on which the at least one second dot        is projected,

the image capturing system, in operation, generating and outputting afirst image signal and a second image signal, the first image signaldenoting a first image of the living body on which the at least onefirst dot is projected, the second image signal denoting a second imageof the living body on which the at least one second dot is projected;and

an arithmetic circuit that, in operation, generates informationconcerning the living body by using the first image signal and thesecond image signal output from the image capturing system and outputsthe generated information, wherein:

the first light includes fifth light having a wavelength longer than orequal to 650 nm and shorter than or equal to 950 nm;

the second light includes sixth light having a wavelength longer than orequal to 650 nm and shorter than or equal to 950 nm; and

the wavelength of the sixth light is longer than the wavelength of thefifth light by 50 nm or more.

In the biological information detection device according to Item 1, theat least one first dot may comprise first dots formed by the firstlight.

In the biological information detection device according to Item 1, theat least one second dot may comprise second dots formed by the secondlight.

In the biological information detection device according to Item 1, theat least one first dot may comprise first dots formed by the firstlight, and the first dots may be arranged in a line.

In the biological information detection device according to Item 1, theat least one second dot may comprise second dots formed by the secondlight, and the econd dots may be arranged in a line.

In the biological information detection device according to Item 1, theat least one first dot may comprise first dots formed by the firstlight, and the first dots may be arranged in an array.

In the biological information detection device according to Item 1, theat least one second dot may comprise second dots formed by the secondlight, and the second dots may be arranged in an array.

[Item 2] In the biological information detection device according toItem 1, the image capturing system may further include

an image sensor having an imaging surface divided into a first region inwhich the first photodetector cells are disposed and a second region inwhich the second photodetector cells are disposed,

a first optical system that, in operation, forms the first image at thefirst region, and

a second optical system that, in operation, forms the second image atthe second region.

[Item 3] In the biological information detection device according toItem 2, the image capturing system may further include

a first bandpass filter that is disposed on a path of the third lightentering the first region and that, in operation, passes the thirdlight, and

a second bandpass filter that is disposed on a path of the fourth lightentering the second region and that, in operation, passes the fourthlight.

[Item 4] In the biological information detection device according toItem 1, the image capturing system may further include

an image sensor

-   -   having an imaging surface at which the first photodetector cells        and the second photodetector cells are disposed, and    -   including first bandpass filters that face the first        photodetector cells and, in operation, pass the third light and        second bandpass filters that face the second photodetector cells        and, in operation, pass the fourth light, and an optical system        that forms the first image and the second image on the imaging        surface.

[Item 5] In the biological information detection system according toItem 1, the image capturing system may further include

an image sensor

-   -   having an imaging surface at which the first photodetector        cells, the second photodetector cells, and third photodetector        cells are disposed, and    -   including first bandpass filters that face the first        photodetector cells and, in operation, pass the third light,        second bandpass filters that face the second photodetector cells        and pass the fourth light, and third bandpass filters that face        the third photodetector cells and, in operation, pass visible        light, and

an optical system that, in operation, forms the first image and thesecond image on the imaging surface, wherein:

the third bandpass filters include color filters having differenttransmission wavelength ranges; and

the image sensor generates and outputs a color image signal by using thethird photodetector cells.

[Item 6] In the biological information detection device according to anyone of Items 1 to 5, the arithmetic circuit may, in operation, generateinformation indicating a blood oxygen saturation of the living body byusing the first image signal and the second image signal and output thegenerated information.

[Item 7] In the biological information detection device according to anyone of Items 1 to 5, the arithmetic circuit may, in operation,

calculate, based on the first image signal and the second image signal,a blood flow and a blood oxygen saturation of the living body,

generate, based on the blood flow and the blood oxygen saturation,information indicating at least one item selected from the groupconsisting of a physical condition, an emotion, and a concentrationdegree of the living body, and

output the generated information.

[Item 8] In the biological information detection device according to anyone of Items 1 to 5, when the first image and the second image includeat least one portion selected from the group consisting of a foreheadportion and a nose portion of the living body,

the arithmetic circuit may, in operation,

calculate, based on the first image signal and the second image signal,a blood flow and a blood oxygen saturation at the at least one portionselected from the group consisting of the forehead portion and the noseportion,

generate, based on a change in the blood flow over time and a change inthe blood oxygen saturation over time, information indicating at leastone item selected from the group consisting of a physical condition, anemotion, and a concentration degree of the living body, and

output the generated information.

[Item 9] In the biological information detection device according to anyone of Items 1 to 5, when the first image and the second image include aforehead portion and a nose portion of the living body,

the arithmetic circuit may, in operation,

calculate, based on the first image signal and the second image signal,a blood flow and a blood oxygen saturation at the forehead portion and ablood flow and a blood oxygen saturation at the nose portion,

generate, based on comparison of a change in the blood flow over timeand a change in the blood oxygen saturation over time at the foreheadportion with a change in the blood flow over time and a change in theblood oxygen saturation over time at the nose portion, informationindicating at least one item selected from the group consisting of aphysical condition, an emotion, and a concentration degree of the livingbody, and

output the generated information.

[Item 10] In the biological information detection device according toany one of Items 1 to 9, each of the first light source and the secondlight source may be a laser light source.

In the biological information detection device according to Item 10, thelaser light source may project a random dot pattern.

[Item 11] In the biological information detection device according toItem 1, the image capturing system may further include

an image sensor having an imaging surface at which the firstphotodetector cells and the second photodetector cells are disposed,

an optical system that, in operation, forms the first image and thesecond image on the imaging surface, and

an adjusting mechanism that, in operation, adjusts focus of the opticalsystem, and

wherein the adjusting mechanism adjusts the focus to maximize contrastof the first image and the second image.

[Item 12] In the biological information detection device according toany one of Items 1 to 11, the arithmetic circuit may, in operation,perform a face recognition process by using the first image signal andthe second image signal, and

generate the information concerning the living body in a case where thefirst image and the second image include at least one portion selectedfrom the group consisting of a forehead, a nose, a mouth, an eyebrow,and hair of the living body.

[Item 13] In the biological information detection device according toany one of Items 1 to 5, the information concerning the living body maybe information concerning at least one item selected from the groupconsisting of a melanin concentration, presence or absence of a spot,and presence or absence of a bruise.

[Item 14] A biological information detection device includes

at least one light source that projects, onto a target, a dot patternformed by light, the target including a living body;

an image capturing system, including photodetector cells, that generatesand outputs an image signal denoting an image of the target on which thedot pattern is projected; and

an arithmetic circuit that is connected to the image capturing systemand that generates and outputs information concerning the living body byusing the image signal output from the image capturing system.

[Item 15] In the biological information detection device according toItem 14, the light may include light of a wavelength longer than orequal to 650 nm and shorter than or equal to 950 nm.

[Item 16] In the biological information detection device according toItem 14 or 15, the information concerning the living body may includeinformation concerning at least one selected from the group consistingof a position of the living body in the image, a heart rate of theliving body, a blood flow of the living body, and a blood oxygensaturation of the living body.

[Item 17] In the biological information detection device according toany one of Items 14 to 16, the image capturing system may furtherinclude

a bandpass filter that passes light of a wavelength range including atleast part of a wavelength range of the light emitted by the lightsource, and

an image capturing element having an imaging surface at which thephotodetector cells are disposed and arranged such that the light thathas passed the bandpass filter enters the imaging surface.

[Item 18] In the biological information detection device according toany one of Items 14 to 17, the arithmetic circuit may determine, basedon a ratio between a standard deviation and an average of pixel valuesof a specific pixel included in the image and pixels neighboring thespecific pixel, whether the living body is located at a positioncorresponding to the specific pixel and may output informationindicating whether the living body is located at the position.

[Item 19] In the biological information detection device according toany one of Items 14 to 18, the arithmetic circuit may generate andoutput information concerning at least one selected from the groupconsisting of a heart rate, a blood pressure, and a blood flow of theliving body, based on a change in a signal in time, the signal beingobtained by performing a low-pass filtering process on at least part ofthe image signal.

Embodiments of the present disclosure will be described in more detailbelow. The following embodiments relate mainly to a biologicalinformation detection device that measures biological information in anon-contact manner, assuming that a face of a person is a living-bodysurface. Note that techniques of the embodiments of the presentdisclosure are applicable not only to a face of a person but also toportions other than the face of a person or to skin of animals otherthan human.

First Embodiment

A system in which a technique of the present disclosure is applied tohuman detection will be described as a first embodiment. Developmentrelating to human detection is underway for the purpose of detectingdisaster victims buried under rubble or the like at a disaster site, forexample. Finding disaster victims within 72 hours from occurrence of adisaster is critical in terms of the survival rate of disaster victims.Accordingly, a simple and stable human detection system is needed. Thehuman detection technology is also utilized in fields of security andtransportation. The human detection technology plays an important roleto find an intruder in the field of security and to detect footpassengers in the field of transportation. There is also an increasingneed for a system capable of selectively detecting a living body(especially, human body) in an image including various constructions orobjects.

FIG. 3A is a diagram illustrating a schematic configuration of a humandetection system according to the first embodiment. As illustrated inFIG. 3A, the human detection system according to the first embodimentincludes a light source 1, an image capturing system (i.e., imagecapturing device or camera) 2, and a computer 20. The light source 1 islocated at a position separate from a living body 3, for example, ahuman body and emits a light beam of a near-infrared wavelength range.The image capturing system 2 is capable of recording an image of aliving-body surface 4 irradiated with light. The computer 20 separatesand measures a component relating to directly reflected light reflectedby the living-body surface 4 and a component relating to scattered lightcaused inside the body by using the captured image and calculatesbiological information on the basis of an intensity of the directlyreflected light and an intensity of the scattered light, and outputs theresulting biological information.

The light source 1 is designed to project a dot pattern onto theliving-body surface 4. Typically, a dot pattern is a collection oftwo-dimensionally arranged small bright points. A one-dimensionallyarranged dot pattern may be used depending on the usage. In the firstembodiment, for example, a random dot pattern laser projector RPP017ESavailable from Osela Inc. in Canada is used as the light source 1. Thislaser light source emits a near-infrared laser beam of 830 nm andprojects a laser dot pattern including 57446 points in a 45°×45° viewingangle.

FIG. 3B is a diagram illustrating an example of a configuration of theimage capturing system (hereinafter, also referred to as a camera) 2 andan example of a generated image. The camera 2 includes a lens 5 and acamera casing 6. The lens 5 may be a set of a plurality of lenses. Thecamera casing 6 includes therein an image sensor 7 and a bandpass filter8 that passes only light of a wavelength of 830 nm±10 nm, which is thewavelength for the light source 1. The image sensor 7 has an imagingsurface at which photodetector cells are disposed.

In the case where the subject is a person, the image sensor 7 obtains animage including a plurality of points each having a brightnesscorresponding to an infrared reflectance at a corresponding position, asillustrated at the center in FIG. 3B. An arithmetic circuit 22 includedin the computer 20 is able to detect only a living body from this imageby performing image signal processing, as illustrated on the right inFIG. 3B. As described before, a living body has a specific opticalproperty called “optical tissue window” for a wavelength range of red tonear-infrared light. Since human skin has a small absorption coefficientand a large scattering coefficient in this wavelength range, light thathas transmitted through the skin surface, which is the living-bodysurface 4, repeats multiple scattering inside the body to scatter andthen exits from a wide area of the living-body surface 4. The livingbody characteristically has a high proportion of scattered lightrelative to directly reflected light in the above wavelength range. Incontrast, objects other than the living body has a high proportion ofdirectly reflected light relative to scattered light. Accordingly, aliving body can be detected based on the ratio between scattered lightand directly reflected light.

FIG. 3C is a block diagram illustrating a configuration of the computer20. The computer 20 includes an input interface (IF) 21, the arithmeticcircuit 22, a memory 24, a control circuit 25, an output interface (IF)23, and a display 26. The input IF 21 is electrically connected to thecamera 2. The arithmetic circuit 22 performs the aforementioned signalprocessing. The memory 24 stores various kinds of data. The controlcircuit 25 controls operations of the entire device. The output IF 23outputs data. The display 26 displays a processing result. Thearithmetic circuit 22 may be an image processing circuit, for example, adigital signal processor (DSP). The control circuit 25 may be anintegrated circuit, for example, a central processing unit (CPU) or amicrocomputer. The control circuit 25 runs a control program stored, forexample, in the memory 24 to perform control, such as providing aninstruction to switch on to the light source 1, an instruction tocapture an image to the camera 2, and an instruction to performcomputation to the arithmetic circuit 22. The control circuit 25 and thearithmetic circuit 22 may be implemented by a single circuit. In thisexample, the computer 20 includes the display 26; however, the display26 may be an external device electrically connected to the computer 20wirelessly or by a cable. The computer 20 may obtain, via acommunication circuit (not illustrated), image information from thecamera 2 located at a remote place.

An example of the human detection method that is carried out usingactual data will be described below.

FIG. 4A illustrates an example of an image obtained by an ordinarycamera that detects visible light. The central part shows a face F of aperson. An image on the left in FIG. 4B is an image that is obtained forthe same scene as that of FIG. 4A by the camera 2 according to the firstembodiment, with the place illuminated with the light source 1 of awavelength of 830 nm. In this image, it is difficult to recognize theface F due to strong reflection by a box B located in the foreground.Accordingly, to detect a living body, the arithmetic circuit 22calculates contrast of directly reflected light and scattered light froma near-infrared image.

FIG. 5 is a diagram illustrating an example of a pixel region used tocalculate contrast. Image data is stored as two-dimensional intensitydata in the memory 24. Here, Pij denotes data of a pixel located at ani-th column in the horizontal (x) direction and a j-th row in thevertical (y) direction. Contrast Cij for this pixel (i, j) is defined asfollows:Cij=Sij/Aij.

Here, Sij and Aij respectively denote a standard deviation and anaverage of pieces of data of pixels in a 7×7 pixel region centered atthe pixel (i, j) and expressed by the corresponding equationsillustrated in FIG. 5. Since the standard deviation Sij decreases as theratio of scattered light to directly reflected light increases, thevalue of the contrast Cij decreases. After repeatedly performing thisprocessing for all pixels, the arithmetic circuit 22 extracts onlypixels for which the value of the contrast Cij is within a predeterminedrange. An example of an image that shows a part of a region where0.2<Cij<0.47 is an image on the right in FIG. 4B. In this image, pixelsfor which the value of the contrast Cij is within the above range areshown in white, and the rest of the pixels are shown in black. The imageindicates that a living body (i.e., the face F) is correctly extracted.

As described above, the arithmetic circuit 22 according to the firstembodiment calculates the contrast Cij, which is a ratio between thestandard deviation and the average of pixel values of a specific pixelincluded in an image and a plurality of pixels neighboring the specificpixel. Based on the value of the contrast Cij, the arithmetic circuit 22is able to determine whether a living body is located at a positioncorresponding to the specific pixel and output information indicatingthe presence or absence of the living body.

According to the first embodiment, a living body hidden behind manyobjects can be efficiently detected by utilizing a specific opticalproperty of a living body. The average and the standard deviation arederived for a 7×7 pixel region to derive a contrast of the image (i.e.,contrast of directly reflected light and scattered light) in thisexample; however, this size of the pixel region is merely an example.The size (i.e., the number of pixels) of the pixel region used tocompute the contrast may be appropriately set in accordance with thedensity of a plurality of dots formed by the light source 1 and theresolution of the camera 2. To suppress variance in the calculationresult, a plurality of (e.g., three or more) dots may be included in apixel region subjected to computation. The accuracy of the calculatedcontrast value improves by increasing the number of pixels included inthe region subjected to computation; however, the resolution of theresulting image of the living body decreases. Accordingly, the number ofpixels included in the region subjected to computation may beappropriately set in accordance with the configuration and usage of thesystem. Likewise, the predetermined contrast range is not limited to0.2<Cij<0.47 and is appropriately set in accordance with theconfiguration and usage of the system.

The system capable of detecting a living body in real time from acaptured image as in the first embodiment is applicable to varioususages. Typical usage examples will be described below.

(1) Finding Disaster Victims at Time of Disaster

Quickly finding disaster victims buried in rubble in response tooccurrence of a natural disaster, such as earthquake, tsunami, or debrisflow, is particularly important to save people's lives. There is “goldentime of 72 hours”, which indicates that the survival rate greatlydecreases after 3 days passes, and it is necessary to quickly finddisaster victims in chaos. The use of the system according to the firstembodiment makes it possible to detect disaster victims hidden behindrubble in real time by capturing an image even in a circumstance whererubble is scattered everywhere. Since the system is small, the systemcan be installed on a drone, for example. This configuration makes itpossible to capture an image while remotely controlling the system at aremote location and to search for survivors even if a disaster site isdifficult to access because of a risk of a secondary disaster.

(2) Monitoring

Surveillance cameras are widely used and contribute to safe and securedaily life. As the number of surveillance cameras increases, it becomesmore important how and who checks a video image captured by thesurveillance cameras. Since it is difficult for a person to check theimage all the time, the image is often used such that the image isaccumulated and the image is checked after occurrence of a problem(crime) to grasp the situation. A utilization method may be adopted inwhich the moment at which a problem occurs is captured from a real-timeimage and the problem is immediately handled. The use of the techniqueaccording to the first embodiment of the present disclosure makes itpossible to construct a system that recognizes a person when the personenters the field of view of a surveillance camera and warns a person incharge to prompt the person in charge to check the image in real time. Asystem can be constructed that frees the person in charge from thenecessity of standing by in front of the monitor of the surveillancecamera and that displays a warning and the image on a mobile terminalcarried by the person in charge upon detecting a person. Such a systemis suitable for monitoring at a backdoor of a warehouse or buildingwhere people rarely appear or a place where access is restricted. Inaddition, for a place, such as a building, where careful monitoring isperformed with many surveillance cameras, highlighting a video image inwhich a certain person is detected may be useful to prevent an unusualsituation from being overlooked or to find out an unusual situation atan early stage.

As for monitoring, development of a method in which a computer performsobject recognition is in progress thanks to the advance in the imagerecognition technology, instead of a traditional method in which aperson judges a monitoring image. For such a usage, a common method isthat an image is transmitted to a host computer and the host computerperforms recognition. However, since this method requires image data betransmitted to the host computer, this method involves issues such as anincreasing amount of communications, a decreasing communication rate,and an increasing load of the host computer. If a surveillance camera iscapable of performing preliminary recognition and judgement on an image,the load for communication, storage, and computation can be greatlyreduced. However, if the recognition is not sufficiently reliable, therecognition may lead to overlooking of an event. Since a person can behighly reliably detected with the human detection method according tothe first embodiment, only a partial image including a person can beselectively transmitted to the host computer upon detecting the person.Consequently, the surveillance system can be efficiently operated.

In addition, the progress in the image recognition technology makes itpossible to identify an individual from an image at a high accuracy. Interms of identification of an individual from an image, a method inwhich an image is transmitted to a host computer and the host computerperforms recognition is commonly used; however, this method alsoinvolves issues relating to load for communication, storage, andcomputation as described above. Specifically, an operation forextracting a face portion for face recognition imposes a heavy loadduring computation. The use of the human detection method according tothe first embodiment allows the face portion to be easily extracted froman image. Accordingly, only the part of the image for the face portioncan be transmitted to the host computer for individual identification,and the load for identifying an individual can be greatly reduced.Further, if the number of people to be identified is limited, asurveillance camera is able to immediately identify an individualwithout using the host computer by registering characteristics of thepeople in the surveillance camera in advance.

(3) Vehicles

Installation of the system according to the first embodiment in avehicle makes it possible to recognize foot passengers on the street andimplement safer driving. Even when a person is hidden behind an objectand is scarcely seen, the system can detect the person and warn thedriver. In terms of automated driving, in a situation where a vehiclecannot stop in response to breaking and an accident is inevitable evenif the vehicle changes the direction to the left and to the right, aquestion about which direction the vehicle should head occurs. In such acase, it is effective to detect living bodies with the system accordingto the first embodiment and to change the heading direction to adirection in which the vehicle can avoid people. Since it is desiredthat the system quickly and highly accurately detect living bodies insuch a usage, the system according to the first embodiment isparticularly suitable.

(4) Switching based on Detection of Person

There is a wide variety of usages in which power is switched on and offby detecting a person. For example, there are usages in which switchingof a device such as an air-conditioner or a light is controlled bydetecting a person in a room, an automatic door is controlled at a highaccuracy, a traffic light for foot passengers is controlled by detectingof a foot passenger at crossing, and brightness of an illumination of avending machine is changed. The first embodiment is applicable to suchusages. The use of the system according to the first embodiment canimplement a sophisticated switch that does not respond to an object orpet but responds only to people. In such a usage, a small humandetection sensor unit including the light source, the camera (imagecapturing system), and the signal processing device (arithmetic circuit)of the system according to the first embodiment may be constructed.

(5) Biometric Authentication

Biometric authentication, such as fingerprint authentication, irisauthentication, and vein authentication, is widely used as a method forauthenticating an individual. With the increasing use of suchauthentication, cases and a risk of spoofing in biometric authenticationare increasing. An image duplication technology, such as copying, hasbeen used in image-based authentication. Recently, with an increasinguse of iris authentication and a three-dimensional printer, a risk ofspoofing using a highly precise duplicate is increasing. As acountermeasure for such a risk, a two-step authentication system iseffective. For example, a method is effective in which ordinarybiometric authentication is performed after checking that a target is aliving body using the human detection system according to the firstembodiment. By checking that the target is a living body using the humandetection system according to the first embodiment, the reliability ofbiometric authentication can be increased.

Second Embodiment

A system in which a technique of the present disclosure is applied tobiological information sensing will be described as a second embodiment.With a growing interest in healthcare, importance of constant biologicalinformation sensing is increasing. A system capable of measuringbiological information in a non-contact manner at all times is essentialnot only at hospitals but also for health management in daily life.

The system according to the second embodiment is capable of monitoringheart rate and a fluctuation of heartbeat in a non-contact manner. Thesystem has a physical configuration substantially the same as that ofthe first embodiment illustrated in FIG. 1. A difference from the firstembodiment is a method of signal processing performed by the arithmeticcircuit 22. Referring to FIGS. 6A and 6B, signal processing inaccordance with the second embodiment will be described below.

FIG. 6A is a diagram illustrating an overview of signal processing inaccordance with the second embodiment. FIG. 6B is a flowchartillustrating a flow of the signal processing. The arithmetic circuit 22performs a known face recognition process on a video image obtained bythe image sensor 7 and extracts a region for a specific portion (e.g.,forehead portion) of the face (step S101). An image on the right end inFIG. 6A illustrates an example of values (pixel values) of pixelsincluded in the extracted region. The arithmetic circuit 22 performs atwo-dimensional lowpass filtering process on at least part of data(i.e., image signal) of the pixel values included in the region toremove a directly reflected light component (i.e., high-frequencycomponent) (step S102). Then, the arithmetic circuit 22 calculates anaverage of the pixel values (i.e., reflected light intensities) in theregion (step S103). The average of the reflected light intensitiesfluctuates over time as illustrated in a graph at a lower part of FIG.6A. Since arterial blood ejected from the heart moves through a bloodvessel while having a fluctuation called a pulse wave, an amount ofabsorbed near-infrared light changes with the pulse. Heart rate can bedetermined from a period of this fluctuation. Further, blood pressure orblood flow can be estimated from an amplitude of the pulse wave (stepS104). As described above, the arithmetic circuit 22 is able to generateinformation concerning at least one of heart rate, blood pressure, andblood flow of a living body, on the basis of a change in a signal overtime, the signal being obtained by performing a lowpass filteringprocess on at least part of an image signal.

It is known that mental stress can be estimated from a fluctuation ofheart rate over time. It is known that a heartbeat interval fluctuateswhen the autonomic nervous system is functioning normally and thatstress makes the fluctuation in heartbeat interval less frequent. Thearithmetic circuit 22 according to the second embodiment is able todetect whether mental stress is imposed or how much stress is imposed,on the basis of a change in the fluctuation of heartbeat interval. Toperform stress sensing in daily life at all times, a non-restraining andnon-contact heartbeat sensing technique such as the second embodiment isessential.

Many methods for monitoring, with an ordinary visible light camera or anear-infrared camera, heartbeat in a non-contact manner have beenproposed. Since separation of the surface reflected light component andthe scattered light component is insufficient in these methods of therelated art, non-contact measurement using such methods is easilyinfluenced by disturbance light and stable and highly accuratemeasurement is difficult. Spatially separating the surface reflectedlight component and the scattered light component as in the secondembodiment enables stable and highly accurate heartbeat measurement tobe performed.

The use of the system according to the second embodiment makes itpossible to monitor heart rate or blood pressure at all times includinga period when the subject is sleeping, without restraining the subject.Consequently, a system can be constructed which monitors the conditionof a patient at a hospital all the time and issues a warning to amedical personnel when something is wrong with the patient. At home, forexample, heart rate of a patient who has the sleep-apnea syndrome can bemonitored at nighttime. Further, since stress sensing can be performedeasily in daily life as described above, people can live a full life.

Third Embodiment

A system for measuring blood oxygen saturation in a non-contact mannerwill be described as a third embodiment. The main role of blood is toreceive oxygen at lungs, carries oxygen to tissues, receives carbondioxide from tissues, and carries carbon dioxide back to lungs.Approximately 15 g of hemoglobin is present in 100 ml of blood.Hemoglobin loaded with oxygen is called oxyhemoglobin (HbO₂), whereashemoglobin not loaded with oxygen is called hemoglobin ordeoxyhemoglobin (Hb). As illustrated in FIG. 2, oxyhemoglobin anddeoxyhemoglobin have different light absorption properties.Oxyhemoglobin absorbs infrared light of wavelengths exceedingapproximately 830 nm relatively well, whereas deoxyhemoglobin absorbsred light (e.g., a wavelength of 660 nm) relatively well. There is nodifference between absorbance of oxyhemoglobin and absorbance ofdeoxyhemoglobin for near-infrared light of a wavelength of 830 nm.Accordingly, in the third embodiment, transmitting light of twowavelengths, that is, 660 nm and 830 nm, are measured. A ratio (oxygensaturation) between two types of hemoglobin can be determined from aratio between transmitting light of infrared light and transmittinglight of red light. Oxygen saturation is a value indicating how muchhemoglobin in blood is loaded with oxygen. The oxygen saturation isdefined by Equation below.Oxygen saturation=C(HbO₂)/[(C(HbO₂)+C(Hb)]×100(%),where C(Hb) denotes a deoxyhemoglobin concentration, and C(HbO₂) denotesan oxyhemoglobin concentration.

A living body includes a non-blood components that absorb light of awavelength of red to near-infrared light; however, a fluctuation inlight absorbance over time results mainly from hemoglobin in arterialblood. Accordingly, a blood oxygen saturation can be measured at a highaccuracy, based on a fluctuation of absorbance. Arterial blood ejectedfrom the heart moves through a blood vessel as a pulse wave, whereasvenous blood does not have a pulse wave. Light radiating a living bodytransmits through the living body after being absorbed at each layer ofthe living body, such as arteries, veins, and non-blood tissues;however, thickness of such tissues other than arteries does not changeover time. Accordingly, scattered light from the living body shows achange in intensity over time in response to a change in the thicknessof an arterial blood layer due to the pulse. This change reflects achange in thickness of the arterial blood layer, and does not containthe influence of venous blood and tissues. Thus, by focusing only on thechange in the scattered light component, information concerning arterialblood can be obtained. Also, heart rate can be determined by measuring aperiod of a change in the component over time.

FIG. 7 is a diagram illustrating a configuration of the system accordingto the third embodiment. The system includes light sources 101 and 102,cameras 201 and 202, and a computer 20. The light sources 101 and 102are two array point light sources that are disposed at positionsseparate from a living body 3 and emit a light beam of a wavelength ofnear-infrared light (e.g. wavelength of 830 nm) and a light beam of awavelength of red light (e.g. wavelength of 660 nm), respectively. Thecameras 201 and 202 are two image capturing systems capable of recordingan image of a living-body surface 4 irradiated with light. The computer20 separates and measures an intensity of directly reflected lightreflected by the living-body surface 4 and an intensity of scatteredlight caused in the body from the obtained image and calculatesbiological information from the intensity of the directly reflectedlight and the intensity of the scattered light. In the third embodiment,the system includes the light sources 101 and 102, which are two arraypoint light sources with different wavelengths, and the cameras 201 and202 respectively corresponding to the light sources 101 and 102 in orderto measure blood oxygen saturation.

FIG. 8 is a diagram illustrating a configuration of the image capturingsystems. Each of the cameras 201 and 202, which are the image capturingsystems, includes a lens 5 and a camera casing 6. The camera casing 6 ofthe camera 201 includes therein an image sensor 7 and a bandpass filter801 that selectively passes near-infrared light (wavelength of 830 nm).The camera casing 6 of the camera 202 includes therein the image sensor7 and a bandpass filter 802 that selectively passes red light(wavelength of 660 nm).

For example, a random dot pattern laser projector RPP017ES availablefrom Osela Inc. in Canada can be used as the light source 101. Thislaser light source is a 830-nm near-infrared laser light source andprojects a laser dot pattern including 57446 points in a 45°×45° viewingangle. For example, a random dot pattern laser projector RPP016ESavailable from Osela Inc. in Canada can be used as the light source 102.This laser light source is a 660-nm red laser light source and projectsa laser dot pattern including 23880 points in a 35°×35° viewing angle.

The computer 20 controls the cameras 201 and 202 and the light sources101 and 102 so that the two cameras 201 and 202 operate together tosimultaneously capture respective images. In this way, images based onlight having two different wavelengths are generated by the cameras 201and 202 as illustrated on the right in FIG. 8, for example.

FIG. 9 is a diagram illustrating transmittance characteristics of thebandpass filters 801 and 802. The bandpass filter 801 has atransmittance characteristic that the transmission center wavelength is830 nm and the bandwidth is 10 nm. The bandpass filter 802 has atransmittance characteristic that the transmission center wavelength is660 nm and the bandwidth is 10 nm. The transmission center wavelengthsof the bandpass filters 801 and 802 respectively match central values ofwavelengths of the light sources 101 and 102. Accordingly, the camera201, which is the image capturing system, obtains an image based onlight of a wavelength of 830 nm, whereas the camera 202, which is theimage capturing system, obtains an image based on light of a wavelengthof 660 nm.

The arithmetic circuit 22 of the computer 20 first performs a known facerecognition process on a video image and extracts a region for aspecific portion (e.g., forehead portion) of the face, as in the secondembodiment. The arithmetic circuit 22 then performs a two-dimensionallowpass filtering process on data of pixels in the region to remove adirectly reflected light component. Then, the arithmetic circuit 22calculates an average of pixel values of the pixels in the region. Thearithmetic circuit 22 performs the above process for each of the camera201 for 830 nm and the camera 202 for 660 nm. The averages thus obtainedindicate intensities of scattered reflected light from the living body3.

FIG. 10 is a diagram illustrating an example of changes in the obtainedscattered reflected light intensities over time. The reflected lightintensity fluctuates over time for both near-infrared light (wavelengthof 830 nm) and red light (wavelength of 660 nm). Here, let li(830) andli(660) respectively denote an intensity of light emitted from the lightsource 101 and an intensity of light emitted from the light source 102at the living-body surface 4, and let Δl(830) and Δl(660) each denote anaverage of the fluctuation component of the scattered reflected lightfrom the living body 3. Blood oxygen saturation SpO₂ is calculated usingEquation below.SpO₂ =a+b*(log(Δl(660)/Δl(660)))/(log(Δl(830)/li(830))),where a and b can be decided upon based on a relationship with measuredvalues obtained by an existing pulse oximeter.

To check the accuracy of the measuring instrument, oxygen saturation ata fingertip instead of the forehead is measured using the systemaccording to the third embodiment. Specifically, oxygen saturation at afingertip is measured while blood flow is stopped by pressuring theforearm at a certain pressure (200 mmHg) using a belt (cuff) used tomeasure the blood pressure.

A commercially available pulse oximeter to which a finger is inserted isattached to the index finger. Oxygen saturation at the middle finger ismeasured in a non-contact manner using the system according to the thirdembodiment. The above values a and b are decided upon by the firstmeasurement, and blood oxygen saturation SpO₂ is measured by the secondand subsequent measurement.

FIG. 11 illustrates a comparison result of the measured values obtainedusing the pulse oximeter and measured values obtained in accordance withthe third embodiment. Since both results substantially match, FIG. 11indicates that measurement is performed accurately. In the method of thethird embodiment, not only blood oxygen saturation but also heart ratecan be simultaneously measured based on the pulse waves illustrated inFIG. 10.

It is known that stress and tiredness can be measured from thefluctuation or a frequency characteristic of the pulse wave. The use ofthe system according to the third embodiment makes it possible toestimate a mental condition, such as stress, and a physical condition ofa subject from the pulse wave in a non-contact manner.

Fourth Embodiment

A method for measuring blood oxygen saturation by using one camera willbe described as a fourth embodiment. In third embodiment, two camerasare used and signals for the light sources with different wavelengthsare obtained by the respective cameras. This method has an advantage inthat existing cameras can be used. However, since it is necessary tocapture images by controlling two cameras to operate together, theconfiguration of the system becomes complicated. Also, since theobtained data is individual pieces of video data for the two cameras,synchronized data processing becomes complicated. To avoid suchcomplexities, in the fourth embodiment, a camera capable ofsimultaneously obtaining data of images for two wavelengths isimplemented.

FIG. 12 is a diagram illustrating a configuration of a biologicalinformation detection device according to the fourth embodiment. Thisbiological information detection device has a structure of a twin-lensreflex camera including two imaging optical systems 201 and 202.Accordingly, herein, such a configuration is referred to as a “stereocamera configuration”. The biological information detection device (alsoreferred to as a “camera”) includes a light source 101 (wavelength of830 nm), which is a first laser point light source, and a light source102 (wavelength of 760 nm), which is a second laser point light source.The light emitted by the light sources 101 and 102 and reflected by aliving body 3 respectively passes through bandpass filters 801 and 802.Then, the propagating direction of the light is bent by mirrors 901 and902 by 90 degrees, and images are formed on imaging surfaces of imagesensors 701 and 702 via lenses 501 and 502, respectively. The bandpassfilters 801 and 802 are narrow-band bandpass filters that pass onlylight of a wavelength of 830±15 nm and light of a wavelength of 760±15nm that correspond to wavelengths of the two light sources 101 and 102,respectively.

In response to pressing of a shutter button 11, the two light sources101 and 102 switch on, and the image sensors 701 and 702 simultaneouslyobtain images of the living body 3. The obtained images are convertedinto images of a stereo-image format by an image processing processor(corresponding to the arithmetic circuit 22 in FIG. 3C), are subjectedto image signal processing, and are accumulated in a storage device(corresponding to the memory 24 in FIG. 3C). The following processing issubstantially the same as that of the second and third embodiments.

According to the fourth embodiment, by configuring an image capturingsystem as one stereo camera, the entire system becomes compact, and theconfiguration of the following signal processing system from imagesignal processing to calculation of oxygen saturation can be simplified.In this way, a simple and high-speed operation can be implemented.

For example, 760 nm and 830 nm, which are in a near-infrared range, canbe used as wavelengths of the two light sources. Since a difference inabsorbance between oxyhemoglobin and deoxyhemoglobin is larger for 660nm used in the second and third embodiments than that for 760 nm, oxygensaturation can be measured more accurately for the wavelength of 660 nm.However, since the wavelength of 660 nm is in a visible light range, theuse of this wavelength may impose a load on the subject. Further, sincelight of a fluorescent lamp and a light-emitting diode (LED)illumination contains a component of the wavelength of 660 nm,measurement is easily affected by ambient light. In the fourthembodiment, the wavelength of 760 nm is selected in consideration ofsuch issues. Since a local absorbance peak of deoxyhemoglobin is at 760nm, it is effective to use a wavelength of 760 nm to 780 nm if thewavelength of the light source having a shorter wavelength is set in thenear-infrared range. The wavelengths used are not limited to the aboveones, and may be appropriately selected in accordance with the usage andthe use environment.

Fifth Embodiment

Another method for measuring blood oxygen saturation by using one camerawill be described as a fifth embodiment. In the fourth embodiment, thestereo camera configuration in which one camera includes two opticalsystems and two image sensors is employed. In the fifth embodiment, asystem is employed which obtains two different images corresponding totwo wavelengths with one image sensor by dividing an image using aplurality of lenses. The configuration according to the fifth embodimentis referred to as a “stereo lens configuration”. The system of thestereo lens configuration will be described with reference to FIG. 13.

FIG. 13 is a cross-sectional view schematically illustrating a part of abiological information detection device according to the fifthembodiment. In the fifth embodiment, an imaging surface of an imagesensor 7 includes a first region 7 a in which first photodetector cellsare disposed and a second region 7 b in which second photodetector cellsare disposed. Although not illustrated in FIG. 13, the biologicalinformation detection device includes, for example, in a camera casing6, two light sources that project dot patterns formed by light of twowavelengths of 830 nm and 760 nm. As illustrated in FIG. 13, a lens 5,which is an optical system, includes therein two sets of lenses 501 and502. The set of lenses 501 and the set of lenses 502, which are theoptical systems, are designed to form respective images at the firstregion 7 a and the second region 7 b on the imaging surface of the imagesensor 7, respectively. Two narrow-band bandpass filters 801 and 802that pass light of 830 nm and 760 nm corresponding to the wavelengths ofthe two light sources are disposed in front of the sets of lenses 501and 502, respectively. Specifically, the bandpass filter 801 is disposedon a path of light entering the first region 7 a, whereas the bandpassfilter 802 is disposed on a path of light entering the second region 7b.

With such a configuration, two images based on light of two wavelengthsfor the same time point can be obtained by using one image sensor 7. Thearithmetic circuit 22 calculates blood oxygen saturation from the twoimages using the method similar to that of the second to fourthembodiments. According to the fifth embodiment, since one image signalinclude information concerning two images corresponding to two differentwavelengths for the same time point, the arithmetic processing becomeseasier.

A result obtained by performing stress sensing by using the system ofthis stereo lens configuration will be described below. As describedabove, a method for detecting, with thermography, a decrease intemperature at a nose portion due to stress (nervousness) orconcentration has been proposed. The blood flow decreases at the noseportion due to a psychological change, and in response to the decreasein the blood flow, temperature decreases at the nose portion. A methodfor detecting such a temperature change with thermography is commonlyperformed. A change in temperature at the face is caused by a change inblood flow. If a change in blood flow can be measured at a highaccuracy, stress sensing can be implemented that is more accurate andmore responsive than in the case of measuring a change in the surfacetemperature, which occurs as a result of a change in blood flow.

FIG. 14A is a diagram illustrating a result of stress sensing performedusing the biological information detection device according to the fifthembodiment. A cold-water load for immersing the right hand into coldwater (ice water) is imposed as stress. For comparison, a temperaturechange is measured using thermography at a nose portion and a cheekportion enclosed by dotted lines in FIG. 14B. FIG. 14A illustratesresults of this measurement. The temperature at the nose portiongradually decreases in about three minutes after the cold-water load isstarted to be imposed and becomes stable after decreasing byapproximately 1.2° C. FIG. 14A indicates that temperature returns to theoriginal level in about three minutes after the load is removed. Incontrast, temperature at the cheek portion is hardly affected by thecold-water load and is stable.

FIG. 14C is a diagram illustrating a change in blood flow and a changein blood oxygen saturation that are obtained by the biologicalinformation detection device according to the fifth embodiment thatemploys the stereo lens configuration. Data for regions corresponding tothe nose portion and the cheek portion, which are denoted bydotted-lines in FIG. 14B, are extracted from data of the blood flow andthe oxygen saturation (SpO₂) at the face. A solid line denotes a changein the blood flow over time, whereas a dotted line denotes a change inthe oxygen saturation (ΔSpO₂) over time. As illustrated in FIG. 14C, theblood flow tends to decrease at the nose portion immediately after acold stimulus is started, which indicates the responsivity with respectto time is high. In contrast, the blood flow hardly changes at the checkportion. A decrease in the oxygen saturation is observed at the noseportion in response to a decrease in the blood flow, whereas the oxygensaturation hardly changes at the cheek portion.

As is apparent from the results, many pieces of data can be obtained bymeasuring blood flow and oxygen saturation at different portions of theface. An emotion, a physical condition, and a concentration degree canbe detected at a high accuracy based on these pieces of data. The changein blood flow due to influence of the autonomic nervous system differsfrom portion to portion of the face. Thus, it is especially important tomeasure a change in blood flow at a specific portion by using thecamera. At that time, the accuracy of the measurement can be increasedby performing, at the same time, measurement at a portion where bloodflow hardly changes and using the result as a reference.

Sixth Embodiment

Another method for measuring blood oxygen saturation by using one camerawill be described as a sixth embodiment.

FIG. 15 is a sectional view schematically illustrating a configurationof a biological information detection device according to the sixthembodiment. The biological information detection device includes astereo adapter 10 attachable to an ordinary lens 5. The stereo adapter10 is an attachment including four mirrors 151, 152, 153, and 154 andtwo bandpass filters 801 and 802. The use of the stereo adapter 10allows two images corresponding to two wavelengths to be formed at twodifferent regions of an imaging surface of an image sensor 7. Thisconfiguration is referred to as a “stereo adapter configuration”.

In the stereo adapter configuration, two different images correspondingto two wavelengths can be obtained by one image sensor 7 by using twosets of facing mirrors. Although not illustrated in FIG. 15, two lightsources that respectively emit light of two wavelengths of 830 nm and760 nm are included in a camera casing 6. The stereo adapter 10 isattached to an end of the lens 5 of the camera. The two sets of mirrors(a set of mirrors 151 and 152 and a set of mirrors 153 and 154) bend thelight path twice to guide the light to the lens 5. The narrow-bandbandpass filters 801 and 802 that respectively pass light of thewavelengths of 830 nm and 760 nm corresponding to the wavelengths of thelight sources are disposed between the lens 5 and the mirrors 151, 152,153, and 154.

This biological information detection device is able to obtain images oftwo wavelengths for the same time point by using one image sensor 7. Thebasic concept is the same as that of the fifth embedment. Since thestereo lens configuration can make the lens small, the entire system canbe advantageously made small. In contrast, with the stereo adapterconfiguration, the entire system becomes larger but a powerful cameralens can be used and the resolution can be improved. Also, lenses ofdifferent magnifications and zoom lenses can be used. The stereo adapterconfiguration advantageously increases the degree of freedom of thesystem.

A study for detecting an emotion of a person by using the biologicalinformation detection device (i.e., camera) according to the sixthembodiment has been carried out. As described in the fifth embodiment, afeeling or emotion such as stress of a human can be stably detectedbased on blood flow. In response to a change in a feeling or emotion ofa person, the autonomic nervous system becomes more active and bloodflow on the skin surface changes. As a result of this change in bloodflow, facial color changes. People detect an emotion and a physicalcondition of a target person from a subtle change in facial colorwithout any difficulty. It is considered that a reason why a greatdoctor can diagnose a patient's physical condition and a cause of adiseases by just looking at the patient's face is that such a doctor canidentify a physical change from the subtle change in color of thepatient's face. In addition, it is said that, when a person who is goodat reading the situation reads a feeling of a counterpart, a subtlechange in facial color as well as a subtle change in facial expressionplay an important role. Further, to make a situation natural and real infields showing remarkable progresses recently, such as game, animation,and computer graphics, studies for subtly changing the facial color of ahuman character are widely carried out. As is apparent from theseexamples, the facial color represents an emotion and a physicalcondition of a person, and an attempt to read a feeling by measuring thefacial color has been studied (for example, Kuroda et al., “Analysis offacial color and skin temperature in emotional change and its synthesisof facial color”, Human interface 1(1), pp. 15-20, 1999). However, suchan attempt to directly measure an emotion from facial color is notsuitable for practical use because stable measurement is difficult. Thisis because a change in facial color differs from person to person, andstable measurement is difficult since a change in facial color is subtleand is strongly influenced by disturbance light and a camera. A methodfor stably and highly accurately detecting an emotion by a measure otherthan measuring a change in facial color is desired.

It is known that facial color of a person is mainly decided by an amountof melanin contained in the skin surface (dermis) and concentrations ofhemoglobin (oxyhemoglobin and deoxyhemoglobin) in blood. Since theamount of melanin does not fluctuate in a short time (changes due to afactor such as aging or tanning), a change in emotion can be stablymeasured by measuring blood flow. In the sixth embodiment, instead ofmeasuring facial color, blood flow of oxyhemoglobin and deoxyhemoglobinthat changes facial color is directly measured to detect a change inemotion. As described in the fifth embodiment, a change in blood flowdiffers from portion to portion of the face because an impact of theinfluence of the autonomic nervous system differs from portion toportion of the face. For example, the nose portion is easily influencedby the autonomic nervous system because lots of arteriovenousanastomosis is located at the nose portion, whereas the forehead portionis hardly influenced by a skin blood vessel contraction nerve. Thearithmetic circuit 22 according to the sixth embodiment determines bloodflows at a plurality of different portions by computation and comparesthe obtained blood flows with each other, thereby being able to detect achange in emotion at a high accuracy.

Measurement of a change in blood flow in response to an emotional changewill be described below. The camera of the stereo adapter configurationillustrated in FIG. 15 is used to measure blood flow. A subject takes aseat in front of the camera, and an image of the subject's face iscaptured with the camera. Color images of the face of the subject areobtained while showing the subject a video image that induces, from thesecured state, feelings such as fear, laughter, surprise, and disgust.An emotional change is read based on a change in scene in the videoimage and the facial change in the color images, and a change in bloodflow at the time of such a change is measured. Blood flow is measured atthe nose portion and the forehead portion as indicated by dotted linesin FIG. 16A.

FIG. 16B is a diagram illustrating a change in total blood flow(oxyhemoglobin and deoxyhemoglobin) over time and a change in thepercentage of oxyhemoglobin blood flow (oxygen saturation) over timewhen an emotion causing laughter is induced. FIG. 16B indicates that thevalue of the total blood flow and the value of the blood oxygensaturation greatly change in response to the emotional change causinglaughter. Similar examinations are performed for other emotions. Thetotal blood flow and an amount of change in the blood oxygen saturationare determined by calculation for the case where other emotions, such assadness, surprise, depression, fear, disgust, anger, concentration, andhappiness, are induced. The same measurement is performed for twelvesubjects. Although there is a variation among individuals, the change inthe total blood flow and the change in the blood oxygen saturation haveshowed the similar tendencies for almost all the subjects. This resultindicates that an emotional change can be detected from at least one ofblood flow and oxygen saturation.

As illustrated in FIG. 16B, a relationship between oxygen saturation andblood flow differs from portion to portion of the face. Accordingly,highly accurate emotion sensing can be performed by determining bloodflow and oxygen saturation at a plurality of portions of the face. Inthe emotion sensing test performed in the sixth embodiment, measurementis performed at three portions, i.e., at the forehead, the cheek, andthe nose. A change in oxygen saturation and a change in blood flow inresponse to an emotional change differ among the forehead, the cheek,and the nose. Accordingly, an emotional change can be detected highlyaccurately by creating in advance a table indicating a relationshipbetween oxygen saturation and an amount of change in blood flow at eachportion and calculating a correlation with the actually measured valuesof oxygen saturation and blood flow.

Seventh Embodiment

A method for measuring blood oxygen saturation by using one camerawithout dividing an image by an optical system will be described as aseventh embodiment. In the second to sixth embodiments, the method fordividing light from two light sources corresponding to two wavelengths,performing sensing, and determining biological information, such asoxygen saturation, by computation has been described. A biologicalinformation detection device according to the seventh embodimentobtains, with an image sensor, two image signals for differentwavelengths without dividing an image.

FIG. 17A is a diagram schematically illustrating a configuration of thebiological information detection device according to the seventhembodiment. This biological information detection device separate twoimages corresponding to two wavelengths not by the optical system but byan image sensor 703. Although illustration of point light sources isomitted in FIG. 17A, two light sources that respectively emit light of awavelength of 830 nm and light of a wavelength of 760 nm are included ina camera casing 6. A bandpass filter 8 that passes light of a wavelengthlonger than or equal to 730 nm and shorter than or equal to 850 nm isdisposed in front of a lens 5 of the camera. The bandpass filter 8 cutsvisible light and infrared light of long wavelengths. Light that haspassed the bandpass filter 8 forms an image on the imaging surface ofthe image sensor 703 via the lens 5. Unlike ordinary image sensors, theimage sensor 703 used in the seventh embodiment includes two kinds ofbandpass filters (hereinafter, also referred to as color filters) thatpass near-infrared light.

FIG. 17B is a diagram illustrating a plurality of color filters thatface a plurality of photodetector cells disposed on the imaging surfaceof the image sensor 703. The image sensor 703includes a plurality ofcolor filters IR1 (also referred to as a first bandpass filter set) thatselectively pass light of a wavelength of 680 nm to 800 nm and aplurality of color filters IR2 (also referred to as a second bandpassfilter set) that selectively pass light of a wavelength of 800 nm orlonger. The color filters IR1 and IR2 are arranged in a checkeredpattern. The lower image in FIG. 17B is a diagram illustrating anexample of wavelength dependency of transmittance of the color filtersIR1 and IR2. The image sensor 703 detects, with a plurality ofphotodetector cells (also referred to as pixels), two images based onlight of 760 nm and 830 nm which are wavelengths of the two lightsources.

The arithmetic circuit 22 (FIG. 3C) separately reads data obtained bythe plurality of photodetector cells of the image sensor 703 for thewavelength of 760 nm and data obtained by the plurality of photodetectorcells for the wavelength of 830 nm. As illustrated in FIG. 17C, thearithmetic circuit 22 generates an image for the wavelength of 760 nmand an image for the wavelength of 830 nm by adding, to thecorresponding data, data for lacking pixels by interpolation. Sincethese two images completely coincide with each other, calculating bloodflow and oxygen saturation from these images is easier than calculationusing two different images. However, since the filtering performance ofthe filters is lower than that achieved in the case where bandpassfilters corresponding to respective light sources are used, there is aconcern about occurrence of color mixing between the light sources inthis method.

Eighth Embodiment

A biological information detection device capable of obtaining not onlytwo images corresponding light sources with two wavelengths withoutdividing an image but also a color image will be described as an eighthembodiment.

FIG. 18A is a diagram illustrating a configuration of the biologicalinformation detection device according to the eighth embodiment.Although illustration of point light sources is omitted also in FIG. 18,two light sources that respectively emit light of a wavelength of 830 nmand light of a wavelength of 760 nm are included in a camera casing 6.In the eighth embodiment, to obtain a color image, no bandpass filter isdisposed in front of a lens 5. Visible light and light emitted by theleaser light sources form images on an imaging surface of an imagesensor 704 via the lens 5. Unlike ordinary image sensors, the imagesensor 704 used in the eighth embodiment includes photodetector cellsthat obtain a color image and two kinds of photodetector cells thatobtain near-infrared images.

FIG. 18B is a diagram illustrating a plurality of bandpass filters (orcolor filters) disposed on the imaging surface of the image sensor 704.The lower image in FIG. 18B illustrates wavelength dependencies ofrelative sensitivities of pixels that face corresponding filters. Asillustrated in FIG. 18B, three types of color filters (R, G, and Bfilters) that respectively pass red light, green light, and blue light,filters IR-1 that pass light of 650 nm or longer, and filters IR-2 thatpass light of 800 nm or longer are arranged on the imaging surface. Thefilters IR-1 are referred to as a first bandpass filter set. The filtersIR-2 are referred to as a second bandpass filter set. R, G, and Bfilters are referred to as a third bandpass filter set. An array inwhich two G filters are disposed diagonally adjacent to each other and Rand B filters are disposed on the opposite diagonal side is the same asthe Bayer array of ordinary image sensors. This image sensor 704 differsfrom ordinary image sensors in that two filters IR-1 and IR-2 arearranged next to a basic unit of four filters arranged in the Bayerarray.

The filter IR1 of the seventh embodiment and the filter IR-1 of theeighth embodiment have different transmission wavelength ranges. Thefilter IR1 of the seventh embodiment is a relatively narrow-band filterthat passes light of a wavelength range from 650 nm to 800 nm. Incontrast, in the eighth embodiment, a filter that pass light of awavelength range of 650 nm or longer is used to simplify themanufacturing process of the image sensor 704. However, theconfiguration is not limited to this one, and the color filter describedin the seventh embodiment can also be used. The filter IR-1 of theeighth embodiment is sensitive to both 760 nm and 830 nm. Accordingly,the arithmetic circuit 22 calculates a signal corresponding to 760 nm bysubtracting the signal of the photodetector cells that face the filterIR-2 from the signal of the photodetector cells that face the filterIR-1 and then determines blood oxygen saturation. Consequently, an image(color image) of red, blue, and green, an image for the wavelength of760 nm, and an image for the wavelength of 830 nm are determined by theimage sensor 704 as illustrated in FIG. 18C.

In this configuration, color mixing is more likely to occur than in theseventh embodiment. However, a color image, blood flow, and blood oxygensaturation can be simultaneously obtained by the simple system using onecamera.

In the eighth embodiment, an example of a configuration of a biologicalinformation sensing camera that uses a multi-spectral sensor thatsupport five wavelengths including two wavelengths in an infrared rangeand three wavelengths (red, blue, and green) in a visible light rangehas been described. With the human-detection-type camera described inthe first embodiment, capturing of a color image and human detection canbe performed through measurement on four wavelengths including onewavelength in an infrared range and three wavelengths (red, blue, andgreen) in a visible light range. The multi-spectral sensor (illustratedin FIG. 18D, for example) having four types of color filterscorresponding to four wavelengths is usable for such a purpose. A colorfilter is disposed such that a near-infrared (IR) pixel is assigned toone pixel out of two green pixels of the Bayer array that is commonlyused in the image sensor. In the eighth embodiment, a camera for asystem that switches on a near-infrared illumination of 850 nm isassumed, and a filter that selectively passes the wavelength of 850 nmis selected as a near-infrared filter. The use of such a camera makes itpossible to use one camera system as an ordinary color camera and ahuman detection camera. Consequently, only one surveillance camera isneeded, and it is easier to extract a color image of a portion in whicha person is detected than in the case where two cameras are used. In theeighth embodiment, a color filter for 850 nm is used; however, thenear-infrared filter may be changed in accordance with the near-infraredlight source used.

Other Embodiments

While the embodiments of the present disclosure have been describedabove by way of example, the present disclosure is not limited to theabove embodiments and can be variously modified. The process describedin each of the above embodiments may be applied to other embodiments.Examples of the other embodiments will be described below.

In the embodiments above, laser light sources are used as the arraypoint light sources; however, light sources of other types may be used.For example, less costly LED light sources may be used. However, lightemitted by the LED light source has a lower straightness than thatemitted by the laser light source and is more likely to spread.Accordingly, a dedicated condensing optical system needs to be used orsome attention needs to be paid, such as restraining a distance betweenan image-capturing target and a camera. The number of array point lightsources is not limited to one or two, and three or more light sourcesmay be used.

The biological information detection device may include an adjustmentmechanism that adjusts focus of the optical system. Such an adjustmentmechanism can be implemented by, for example, a motor (not illustrated)and the control circuit 25 illustrated in FIG. 3C. Such an adjustmentmechanism adjusts focus of the optical system to maximize contrast of adot pattern image projected onto a target by the light source. With thisconfiguration, accuracy of calculation of contrast described in thefirst embodiment improves.

The arithmetic circuit 22 may perform a face recognition process usingimage signals output by the image sensor and output informationconcerning a living body if the image includes at least one of aforehead, a nose, a mouth, an eyebrow, and hair. A system may beimplemented that displays an error if none of the forehead, the nose,the mouth, the eyebrow, and the hair of a living body is included in theimage.

The arithmetic circuit 22 may generate information concerning theepidermis including information concerning at least one of a melaninconcentration, presence or absence of a spot, and presence or absence ofa bruise on the basis of the image signal. As described above, theepidermis contains melanin that strongly absorbs light. A spot and abruise are caused as a result of an increase in melanin. Accordingly, amelanin concentration, a spot, and a bruise can be detected based on anintensity distribution of light from the living-body surface.

In the present disclosure, the double camera configuration using twocameras (FIG. 7), the stereo camera configuration (FIG. 12) in which onecamera includes two optical systems and two image sensors, the stereolens configuration (FIG. 13) using two sets of lenses and one imagesensor, the stereo adapter configuration (FIG. 15) using a lens adapter,one lens, and one image sensor, the configuration (FIG. 17A and FIG.18A) that divides an image using the image sensor have been described.As already described, since each configuration has advantages anddrawbacks, an appropriate configuration can be selected in accordancewith the usage.

As described above, according to the embodiments of the presentdisclosure, not only heart rate and blood flow but also blood oxygensaturation can be measured without restraining a subject and withoutplacing a detection device, such as a sensor, in contact with thesubject. An emotion or a physical condition of the subject can also beestimated from measured values of blood flow and oxygen saturation atdifferent portions of the subject.

What is claimed is:
 1. A biological information detection devicecomprising: a first light source that, in operation, projects, onto aliving body, at least one first dot formed by first light; a secondlight source that, in operation, projects, onto the living body, atleast one second dot formed by second light; an image capturing systemincluding first photodetector cells that, in operation, detect thirdlight from the living body on which the at least one first dot isprojected, and second photodetector cells that, in operation, detectfourth light from the living body on which the at least one second dotis projected, the image capturing system, in operation, generating andoutputting a first image signal and a second image signal, the firstimage signal denoting a first image of the living body on which the atleast one first dot is projected, the second image signal denoting asecond image of the living body on which the at least one second dot isprojected; and an arithmetic circuit that, in operation, generatesinformation concerning the living body by using the first image signaland the second image signal output from the image capturing system andoutputs the generated information, wherein: the first light includesfifth light having a wavelength longer than or equal to 650 nm andshorter than or equal to 950 nm; the second light includes sixth lighthaving a wavelength longer than or equal to 650 nm and shorter than orequal to 950 nm; and the wavelength of the sixth light is longer thanthe wavelength of the fifth light by 50 nm or more.
 2. The biologicalinformation detection device according to claim 1, wherein the imagecapturing system further includes an image sensor having an imagingsurface divided into a first region in which the first photodetectorcells are disposed and a second region in which the second photodetectorcells are disposed, a first optical system that, in operation, forms thefirst image at the first region, and a second optical system that, inoperation, forms the second image at the second region.
 3. Thebiological information detection device according to claim 2, whereinthe image capturing system further includes a first bandpass filter thatis disposed on a path of the third light entering the first region andthat, in operation, passes the third light, and a second bandpass filterthat is disposed on a path of the fourth light entering the secondregion and that, in operation, passes the fourth light.
 4. Thebiological information detection device according to claim 1, whereinthe image capturing system further includes an image sensor having animaging surface at which the first photodetector cells and the secondphotodetector cells are disposed, and including first bandpass filtersthat face the first photodetector cells and, in operation, pass thethird light and second bandpass filters that face the secondphotodetector cells and, in operation, pass the fourth light, and anoptical system that, in operation, forms the first image and the secondimage on the imaging surface.
 5. The biological information detectiondevice according to claim 1, wherein the image capturing system furtherincludes an image sensor having an imaging surface at which the firstphotodetector cells, the second photodetector cells, and thirdphotodetector cells are disposed, and including first bandpass filtersthat face the first photodetector cells and, in operation, pass thethird light, second bandpass filters that face the second photodetectorcells and, in operation, pass the fourth light, and third bandpassfilters that face the third photodetector cells and, in operation, passvisible light, and an optical system that, in operation, forms the firstimage and the second image on the imaging surface, wherein: the thirdbandpass filters include color filters having different transmissionwavelength ranges; and the image sensor, in operation, generates andoutputs a color image signal by using the third photodetector cells. 6.The biological information detection device according to claim 1,wherein the arithmetic circuit, in operation, generates informationindicating a blood oxygen saturation of the living body by using thefirst image signal and the second image signal and outputs the generatedinformation.
 7. The biological information detection device according toclaim 1, wherein the arithmetic circuit, in operation, calculates, basedon the first image signal and the second image signal, a blood flow anda blood oxygen saturation of the living body, generates, based on theblood flow and the blood oxygen saturation, information indicating atleast one item selected from the group consisting of a physicalcondition, an emotion, and a concentration degree of the living body,and outputs the generated information.
 8. The biological informationdetection device according to claim 1, wherein when the first image andthe second image include at least one portion selected from the groupconsisting of a forehead portion and a nose portion of the living body,the arithmetic circuit, in operation, calculates, based on the firstimage signal and the second image signal, a blood flow and a bloodoxygen saturation at the at least one portion selected from the groupconsisting of the forehead portion and the nose portion, generates,based on a change in the blood flow over time and a change in the bloodoxygen saturation over time, information indicating at least one itemselected from the group consisting of a physical condition, an emotion,and a concentration degree of the living body, and outputs the generatedinformation.
 9. The biological information detection device according toclaim 1, wherein when the first image and the second image include aforehead portion and a nose portion of the living body, the arithmeticcircuit, in operation, calculates, based on the first image signal andthe second image signal, a blood flow and a blood oxygen saturation atthe forehead portion and a blood flow and a blood oxygen saturation atthe nose portion, generates, based on comparison of a change in theblood flow over time and a change in the blood oxygen saturation overtime at the forehead portion with a change in the blood flow over timeand a change in the blood oxygen saturation over time at the noseportion, information indicating at least one item selected from thegroup consisting of a physical condition, an emotion, and aconcentration degree of the living body, and outputs the generatedinformation.
 10. The biological information detection device accordingto claim 1, wherein each of the first light source and the second lightsource is a laser light source.
 11. The biological information detectiondevice according to claim 1, wherein the image capturing system furtherincludes an image sensor having an imaging surface at which the firstphotodetector cells and the second photodetector cells are disposed, anoptical system that, in operation, forms the first image and the secondimage on the imaging surface, and an adjusting mechanism that, inoperation, adjusts focus of the optical system, and wherein theadjusting mechanism adjusts the focus to maximize contrast of the firstimage and the second image.
 12. The biological information detectionmethod according to claim 1, wherein the arithmetic circuit, inoperation, performs a face recognition process by using the first imagesignal and the second image signal, and generates the informationconcerning the living body in a case where the first image and thesecond image include at least one portion selected from the groupconsisting of a forehead, a nose, a mouth, an eyebrow, and hair of theliving body.
 13. The biological information detection device accordingto claim 1, wherein the information concerning the living body isinformation concerning at least one item selected from the groupconsisting of a melanin concentration, presence or absence of a spot,and presence or absence of a bruise.